Graphics processors generate images for head mounted displays (HMDs) in augmented reality (AR) and virtual reality (VR) applications using a sequence of programmable shaders and fixed function hardware blocks. For example, a 3-D model of an object that is visible in a frame is represented by a set of primitives such as triangles, other polygons, or patches which are processed in a graphics pipeline to produce values of pixels for display to a user. Pixels are typically rendered in window space with uniform pixel spacing across the window. Consequently, the pixel density per subtended angle increases with pixel distance from a central gaze direction of a user or a center of a lens in the HMD. The pixels are sampled at regular intervals by a scan converter to produce a rectangular image. A post processing step distorts the rectangular image so that the image appears regular when a user views the image through a set of display lenses, e.g., when the user is wearing an HMD.